An homotopy method for 𝓵p regression provably beyond self-concordance and in input-sparsity time

We consider the problem of linear regression where the l2 norm loss (i.e., the usual least squares loss) is replaced by the lp norm. We show how to solve such problems up to machine precision in O∗(n|1/2−1/p|) (dense) matrix-vector products and O∗(1) matrix inversions, or alternatively in O∗(n|1/2−1/p|) calls to a (sparse) linear system solver. This improves the state of the art for any p 6∈ {1, 2,+∞}. Furthermore we also propose a randomized algorithm solving such problems in input sparsity time, i.e., O∗(Z + poly(d)) where Z is the size of the input and d is the number of variables. Such a result was only known for p = 2. Finally we prove that these results lie outside the scope of the Nesterov-Nemirovski’s theory of interior point methods by showing that any symmetric self-concordant barrier on the lp unit ball has self-concordance parameter Ω̃(n).